Deep Reinforcement Learning for Swarm Systems

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, these methods rely on a concatenation of agent states to represent the information content required for decentralized decision making. However, concatenation scales poorly to swarm systems with a large number of homogeneous agents as it does not exploit the fundamental properties inherent to these systems: (i) the agents in the swarm are interchangeable and (ii) the exact number of agents in the swarm is irrelevant. Therefore, we propose a new state representation for deep multi-agent RL based on mean embeddings of distributions. We treat the agents as samples of a distribution and use the empirical mean embedding as input for a decentralized policy. We define different feature spaces of the mean embedding using histograms, radial basis functions and a neural network learned end-to-end. We evaluate the representation on two well known problems from the swarm literature (rendezvous and pursuit evasion), in a globally and locally observable setup. For the local setup we furthermore introduce simple communication protocols. Of all approaches, the mean embedding representation using neural network features enables the richest information exchange between neighboring agents facilitating the development of more complex collective strategies.Comment: 31 pages, 12 figures, version 3 (published in JMLR Volume 20

NEFI: Network Extraction From Images

Networks and network-like structures are amongst the central building blocks of many technological and biological systems. Given a mathematical graph representation of a network, methods from graph theory enable a precise investigation of its properties. Software for the analysis of graphs is widely available and has been applied to graphs describing large scale networks such as social networks, protein-interaction networks, etc. In these applications, graph acquisition, i.e., the extraction of a mathematical graph from a network, is relatively simple. However, for many network-like structures, e.g. leaf venations, slime molds and mud cracks, data collection relies on images where graph extraction requires domain-specific solutions or even manual. Here we introduce Network Extraction From Images, NEFI, a software tool that automatically extracts accurate graphs from images of a wide range of networks originating in various domains. While there is previous work on graph extraction from images, theoretical results are fully accessible only to an expert audience and ready-to-use implementations for non-experts are rarely available or insufficiently documented. NEFI provides a novel platform allowing practitioners from many disciplines to easily extract graph representations from images by supplying flexible tools from image processing, computer vision and graph theory bundled in a convenient package. Thus, NEFI constitutes a scalable alternative to tedious and error-prone manual graph extraction and special purpose tools. We anticipate NEFI to enable the collection of larger datasets by reducing the time spent on graph extraction. The analysis of these new datasets may open up the possibility to gain new insights into the structure and function of various types of networks. NEFI is open source and available http://nefi.mpi-inf.mpg.de

Remarks on Category-Based Routing in Social Networks

It is well known that individuals can route messages on short paths through social networks, given only simple information about the target and using only local knowledge about the topology. Sociologists conjecture that people find routes greedily by passing the message to an acquaintance that has more in common with the target than themselves, e.g. if a dentist in Saarbr\"ucken wants to send a message to a specific lawyer in Munich, he may forward it to someone who is a lawyer and/or lives in Munich. Modelling this setting, Eppstein et al. introduced the notion of category-based routing. The goal is to assign a set of categories to each node of a graph such that greedy routing is possible. By proving bounds on the number of categories a node has to be in we can argue about the plausibility of the underlying sociological model. In this paper we substantially improve the upper bounds introduced by Eppstein et al. and prove new lower bounds.Comment: 21 page

Guided Deep Reinforcement Learning for Swarm Systems

In this paper, we investigate how to learn to control a group of cooperative agents with limited sensing capabilities such as robot swarms. The agents have only very basic sensor capabilities, yet in a group they can accomplish sophisticated tasks, such as distributed assembly or search and rescue tasks. Learning a policy for a group of agents is difficult due to distributed partial observability of the state. Here, we follow a guided approach where a critic has central access to the global state during learning, which simplifies the policy evaluation problem from a reinforcement learning point of view. For example, we can get the positions of all robots of the swarm using a camera image of a scene. This camera image is only available to the critic and not to the control policies of the robots. We follow an actor-critic approach, where the actors base their decisions only on locally sensed information. In contrast, the critic is learned based on the true global state. Our algorithm uses deep reinforcement learning to approximate both the Q-function and the policy. The performance of the algorithm is evaluated on two tasks with simple simulated 2D agents: 1) finding and maintaining a certain distance to each others and 2) locating a target.Comment: 15 pages, 8 figures, accepted at the AAMAS 2017 Autonomous Robots and Multirobot Systems (ARMS) Worksho

On efficiency and reliability in computer science

Efficiency of algorithms and robustness against mistakes in their implementation or uncertainties in their input has always been of central interest in computer science. This thesis presents results for a number of problems related to this topic. Certifying algorithms enable reliable implementations by providing a certificate with their answer. A simple program can check the answers using the certificates. If the the checker accepts, the answer of the complex program is correct. The user only has to trust the simple checker. We present a novel certifying algorithm for 3-edge-connectivity as well as a simplified certifying algorithm for 3-vertex-connectivity. Occasionally storing the state of computations, so called checkpointing, also helps with reliability since we can recover from errors without having to restart the computation. In this thesis we show how to do checkpointing with bounded memory and present several strategies to minimize the worst-case recomputation. In theory, the input for problems is accurate and well-defined. However, in practice it often contains uncertainties necessitating robust solutions. We consider a robust variant of the well known k-median problem, where the clients are grouped into sets. We want to minimize the connection cost of the expensive group. This solution is robust against which group we actually need to serve. We show that this problem is hard to approximate, even on the line, and evaluate heuristic solutions.Effizienz von Algorithmen und Zuverlässigkeit gegen Fehlern in ihrer Implementierung oder Unsicherheiten in der Eingabe ist in der Informatik von großem Interesse. Diese Dissertation präsentiert Ergebnisse für Probleme in diesem Themenfeld. Zertifizierende Algorithmen ermöglichen zuverlässige Implementierungen durch Berechnung eines Zertifikats für ihre Antworten. Ein einfaches Programm kann die Antworten mit den Zertifikaten überprüfen. Der Nutzer muss nur dem einfachen Programms vertrauen. Wir präsentieren einen neuen zertifizierenden Algorithmus für 3-Kantenzusammenhang und einen vereinfachten zertifizierenden Algorithmus für 3-Knotenzusammenhang. Den Zustand einer Berechnung gelegentlich zu speichern, sog. Checkpointing, verbessert die Zuverlässigkeit. Im Fehlerfall kann ein gespeicherter Zustand wiederhergestellt werden ohne die Berechnung neu zu beginnen. Wir zeigen Strategien für Checkpointing mit begrenztem Speicher, die die Neuberechnungszeit minimieren. Traditionell sind die Eingaben für Probleme präzise und wohldefiniert. In der Praxis beinhalten die Eingaben allerdings Unsicherheiten und man braucht robuste Lösungen. Wir betrachten eine robuste Variante des k-median Problem. Hier sind die Kunden in Gruppen eingeteilt und wir möchten die Kosten der teuersten Gruppe minimieren. Dies macht die Lösung robust gegenüber welche der Gruppen letztlich bedient werden soll. Wir zeigen, dass dieses Problem schwer zu approximieren ist und untersuchen Heuristiken

Public service motivation, prosocial motivation and altruism : towards disentanglement and conceptual clarity

Research on public service motivation (PSM) has made great strides in terms of study output. Given the enormous scholarly attention on PSM, it is surprising that considerable conceptual ambiguities and overlaps with related concepts such as prosocial motivation, and altruism still remain. This study addresses this issue by systematically carving out the differences and similarities between these concepts. Taking this approach, this study clarifies the conceptual space of both PSM and the other concepts. Using data from semi-structured interviews with police officers, it is illustrated that PSM and prosocial motivation are different types of motivation leading to different types of prosocial behaviour

Attracting future civil servants with public values? An experimental study on employer branding

A frequently cited recommendation of public service motivation (PSM) research is to use PSM in the context of HR marketing. However, empirical evidence demonstrating the usefulness of addressing PSM in the recruitment process is limited. Moreover, we know little about the relative importance of PSM for public employers’ attractiveness. We address this gap using an experimental research design to investigate whether public service motivated individuals differ from extrinsically motivated individuals in terms of their attraction to organizations that emphasize either “traditional” public or private values in their employer branding. Our findings indicate that public service motivated individuals are attracted neither to public nor to private values in employer branding. Furthermore, individuals with very high levels of extrinsic motivation are more attracted to private values employer branding than to public values employer branding and to the control group

Vernalization shapes shoot architecture and ensures the maintenance of dormant buds in the perennial Arabis alpina

Perennials have a complex shoot architecture with axillary meristems organized in zones of differential bud activity and fate. This includes zones of buds maintained dormant for multiple seasons and used as reservoirs for potential growth in case of damage. The shoot of Arabis alpina, a perennial relative of Arabidopsis thaliana, consists of a zone of dormant buds placed between subapical vegetative and basal flowering branches. This shoot architecture is shaped after exposure to prolonged cold, required for flowering.To understand how vernalization ensures the maintenance of dormant buds, we performed physiological and transcriptome studies, followed the spatiotemporal changes of auxin, and generated transgenic plants.Our results demonstrate that the complex shoot architecture in A. alpina is shaped by its flowering behavior, specifically the initiation of inflorescences during cold treatment and rapid flowering after subsequent exposure to growth-promoting conditions. Dormant buds are already formed before cold treatment. However, dormancy in these buds is enhanced during, and stably maintained after, vernalization by a BRC1-dependent mechanism. Post-vernalization, stable maintenance of dormant buds is correlated with increased auxin response, transport, and endogenous indole-3-acetic acid levels in the stem.Here, we provide a functional link between flowering and the maintenance of dormant buds in perennials

Deep Reinforcement Learning for Swarm Systems

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, the observation vector for decentralized decision making is represented by a concatenation of the (local) information an agent gathers about other agents. However, concatenation scales poorly to swarm systems with a large number of homogeneous agents as it does not exploit the fundamental properties inherent to these systems: (i) the agents in the swarm are interchangeable and (ii) the exact number of agents in the swarm is irrelevant. Therefore, we propose a new state representation for deep multi-agent RL based on mean embeddings of distributions, where we treat the agents as samples and use the empirical mean embedding as input for a decentralized policy. We define different feature spaces of the mean embedding using histograms, radial basis functions and neural networks trained end-to-end. We evaluate the representation on two well-known problems from the swarm literature in a globally and locally observable setup. For the local setup we furthermore introduce simple communication protocols. Of all approaches, the mean embedding representation using neural network features enables the richest information exchange between neighboring agents, facilitating the development of complex collective strategies

Public service motivation: State of the art and conceptual cleanup

Public service motivation is an increasingly researched and, at the same time, hotly debated concept in the field of public management and public administration. It refers to the motivation people have to contribute to society. This chapter provides an overview of what has happened so far in this field since the introduction of the concept in the 1980s and 1990s, with a particular focus on the role of the research community. In this overview, causes, consequences, and related theories are identified. The chapter also establishes gaps in the literature and issues that remain unresolved. In so doing, we carry out a conceptual cleanup by positioning the subject alongside related but different concepts such as intrinsic motivation, altruism, and prosocial motivation